Many organizations utilize complex software systems, such as Enterprise Resource Planning (ERP) systems, for conducting almost all aspects of their business. Thus, it is important that the software systems perform as expected. Organizations often devote many resources for the purpose of testing and validating the performance of their software systems. This task is often daunting given the complexity of the software systems, the large number of software modules they may include, and the wide array of possibilities for customizations that may be done to the software modules.
Different organizations may utilize different software modules and also customize them differently. However, it is often the case that software systems belonging to different organizations end up utilizing many software modules that are the same or similar, and/or software modules that involve similar customizations. For example, software systems belonging to different organizations in the same field of operations may involve similar customizations that are related to the field of operations. Additionally, software systems of different organizations may involve similar updates, such as when a new build of a generic software module is released. Thus, despite organization-specific differences in software systems of different organization, it is often the case that testers belonging to the different organizations end up running the same, or quite similar, tests for testing at least some of the components of their respective systems.
Current testing approaches adopted by many organizations require each organization to devise its own testing suite. Thus, each organization needs to learn which tests are effective, which components of the software system should be tested, and how to do so in a cost-effective way. Gaining this knowledge may require much effort and experience; in the meantime, testing the software systems may be a less effective and prolonged process. However, were organizations able to utilize each other's testing-related knowledge, which is in a sense a wisdom of the (testing) crowd, they might be able to come up with a more effective and efficient testing plan. That being said, there are many obstacles in the way of harnessing the wisdom of the crowd when it comes to testing. For one, software systems belonging to different organizations are each customized for their respective organizations. Thus, it is not likely that a test devised for a first organization will run “as is” on a system belonging to a second organization. Additionally, tests used to test a system belonging to an organization often contain proprietary data related to the organization. Thus, organizations are not likely to share their testing data if it means that in the process, their proprietary data is at risk of being leaked to an unauthorized party.